Attention Meets Perturbations: Robust and Interpretable Attention With Adversarial Training

Although attention mechanisms have been applied to a variety of deep learning models and shown improve the prediction performance, it has reported be vulnerable perturbations mechanism. To overcome vulnerability in mechanism, we are inspired by adversarial training (AT), which is powerful regularization technique for enhancing robustness models. In this paper, propose general natural language processing tasks, including AT (Attention AT) more interpretable iAT). The proposed techniques improved performance model interpretability exploiting with AT. particular, Attention iAT boosts those advantages introducing perturbation, enhances difference sentences. Evaluation experiments ten open datasets revealed that mechanisms, especially iAT, demonstrated (1) best nine out tasks (2) (i.e., resulting correlated strongly gradient-based word importance) all tasks. Additionally, (3) much less dependent on perturbation size